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SUMMARY 


\L 


A  simulation  study  was  undertaken  to  determine  the  effect  of  traffic-sensor 
noise  on  the  ability  of  a  pilot  to  perform  an  in-trail  spacing  task.  The  tests  were 
conducted  in  a  fixed-base  cockpit  simulator  configured  as  a  current-generation  trans¬ 
port  aircraft,  with  an  electronic  traffic  display  provided  in  the  weather-radarscope 
location.  The  true  positions  of  the  traffic  were  perturbed  in  both  relative  range 
and  azimuth  by  random  errors  to  simulate  traffic-sensor  noise  associated  with  an 
onboard  sensor.  The  evaluation  task  involved  simulated  instrument  approaches  into 
a  terminal  area  while  maintaining  self-separation  on  a  lead  aircraft.  Separation 
performance  data  and  pilot  subjective  ratings  and  comments  were  obtained  during  the 
study. 

The  results  of  the  separation  data  indicate  that  displayed  traffic  position 
errors,  having  standard-deviation  values  up  to  0.3-n.mi.  range  and  86  azimuth,  had 
negligible  effect  on  the  spacing  performance  achieved  by  the  pilots.  Speed  profiles 
of  the  lead  aircraft,  display  of  the  lead  aircraft  groundspeed,  and  individual  pilot 
techniques  were  found  to  significantly  affect  the  mean  spacing  performance.  Pilot 
comments  and  ratings  indicated  that  despite  the  ability  to  successfully  use  the  traf¬ 
fic  position  data  with  high  sensor  noise  levels,  the  pilots  objected  to  even  small 
errors  in  the  displayed  traffic  location.  Position  errors  with  standard  deviations 
of  0.1-n.mi.  range  and  2°  azimuth  were  rated  as  the  maximum  noise  values  which  were 
acceptable  to  the  pilots  for  performing  the  self-spacing  task.  High  mental  workload 
and  confusion  over  the  true  traffic  location  were  cited  as  the  reasons  for  objecting 
to  the  displayed  traffic  position  errors. 


INTRODUCTION 

Future  growth  of  air  transportation  is  dependent  on  the  ability  of  the  air- 
traffic  control  (ATC)  system  to  accommodate  the  increasing  demand  for  capacity  at  the 
major  high-density  terminal  airports.  Currently,  many  airports  are  capacity-limited 
during  peak  operating  periods,  resulting  in  costly  aircraft  delays  and  high  workload 
levels  for  air-traffic  controllers.  One  method  that  has  been  proposed  to  reduce 
controller  workload  and  increase  airport  capacity  is  to  provide  traffic  information 
in  the  cockpit  to  allcw  greater  pilot  participation  in  the  ATC  process  and,  possibly, 
permit  the  use  of  more  efficient  procedures.  This  concept  was  first  proposed  in 
the  1940's  (ref.  1)»  however,  early  efforts  involving  TV  broadcast  of  the  control¬ 
lers'  radarscope  were  abandoned  because  of  numerous  technical  deficiencies.  Recent 
advances  in  computer  technology,  digital  data  links,  and  electronic  flight  displays 
have  resulted  in  renewed  interest  in  the  concept. 

Numerous  simulation  studies,  most  notably  the  efforts  by  the  Massachusetts 
Institute  of  Technology  in  the  early  1970's  (ref.  2),  have  demonstrated  pilot  accep¬ 
tance  of  traffic  information  and  have  identified  several  possible  benefits  associated 
with  active  use  of  traffic-situation  displays.  Currently,  the  National  Aeronautics 
and  Space  Administration  (NASA)  and  the  Federal  Aviation  Administration  (FAA)  have 
undertaken  a  joint  program  to  explore  potential  cockpit  display  of  traffic  informa¬ 
tion  (CDTI)  applications  under  realistic  environmental  and  workload  conditions.  As  a 
part  of  this  program,  NASA  Langley  Research  Center  is  investigating  CDTI  applications 
in  the  operation  of  current,  conventionally  equipped  transport  aircraft  through  the 


use  of  piloted  simulation  studies.  An  important  consideration  in  these  studies  is 
the  impact  of  traffic-sensor  characteristics  on  the  ability  of  pilots  to  effectively 
utilize  the  CDTI. 

The  primary  objective  of  this  study  was  to  determine  the  effect  of  traffic¬ 
sensor  errors  on  the  ability  of  a  pilot  to  perform  an  in-trail  spacing  task.  The 
tests  were  conducted  in  a  fixed-based  cockpit  simulator  configured  as  a  current- 
generation  transport  aircraft  with  an  electronic  traffic  display  provided  in  the 
weather-radarscope  location  (fig.  1).  The  true  positions  of  the  traffic  were  per¬ 
turbed  in  both  range  and  azimuth  by  random  errors  to  simulate  traffic-sensor  noise 
associated  with  an  onboard  sensor.  Range  noise  errors  (with  standard-deviation 
values  of  up  to  0.3  n.mi.)  and  azimuth  noise  errors  (with  standard-deviation  values 
up  to  8°)  were  evaluated. 

The  primary  pilot  task  for  this  study  was  to  achieve  and  maintain  specified 
spacing  intervals  behind  a  cockpit-displayed  lead  aircraft  while  conducting  a  simu¬ 
lated  approach.  Two  pilots  each  flew  54  approaches  into  a  simulated  Denver-Stapleton 
environment  (fig.  2)  with  varying  error  levels  present  in  the  traffic  display.  Data 
were  taken  in  the  form  of  quantitative  performance  measures  as  well  as  subjective 
pilot  ratings  and  comments. 


RESEARCH  SYSTEM 
Simulator  Description 

This  study  was  conducted  utilizing  a  fixed-base  cockpit  simulator  configured  as 
a  conventional,  two-engine  jet  transport  aircraft  (fig.  1).  The  four  throttle  con¬ 
trols  present  in  the  cockpit  were  mechanically  pinned  together  in  pairs  to  represent 
the  two-engine  configuration.  The  aircraft  dynamics  modeled  for  the  simulation  were 
those  of  a  Boeing  737.  Nonlinear  aerodynamic  data  and  atmospheric  effects  were 
included  in  the  simulation  model.  The  host  computer  for  the  simulation  was  a  Control 
Data  CYBER  175  system,  which  contained  the  aircraft  dynamics,  navigation,  and  flight 
director  algorithms.  Conventional  navigation  instruments,  which  included  horizontal 
situation  indicators,  flight  director,  and  distance  measuring  equipment  (DME) ,  were 
provided  in  the  cockpit.  Flight  instrumentation  consisted  of  standard  instruments 
required  for  manual  flight  control;  however,  no  autopilot  or  automatic  flight  control 
systems  were  provided  to  the  pilot.  In  addition,  no  attempt  was  made  to  duplicate 
any  specific  aircraft  cockpit  configuration  or  control-force  feel  characteristics. 


Traffic  Generation  Scheme 

The  displayed  traffic  was  generated  from  data  previously  recorded  using  the 
Langley  Real-Time  Simulation  System.  Specifically,  the  traffic  data  were  created  by 
using  a  piloted  simulation  capability  wherein  flights  were  made  along  each  of  the 
routes  that  corresponded  to  the  airway  structure  prescribed  by  the  test  scenarios. 
These  individual  flights  were  recorded  and  then  merged  into  a  set  of  data  that  was 
position  and  time  correlated.  The  output  of  these  merged  data  was  the  representation 
of  numerous  airplanes  following  several  flight  paths.  This  traffic-generation  tech¬ 
nique  was  developed  for  use  in  the  study  described  in  reference  3.  A  description  of 
the  actual  traffic  scenarios  used  in  this  study  is  contained  in  the  section  of  this 
report  entitled  "Traffic  Profiles." 


EXPERIMENT  DESIGN 


CDTI  Display 

The  display  used  as  the  CDTI  for  this  study  was  a  monochrome  cathode-ray  tube 
(CRT)  located  behind  the  throttle  quadrant  as  shown  in  figure  1.  This  location 
corresponds  to  the  normal  location  for  a  weather-radar  display  on  most  conventionally 
equipped  transport  aircraft.  Although  the  CRT  measured  10  in.  across  the  diagonal, 
an  opaque  mask  was  used  to  reduce  the  display  size  to  5  in.  high  by  4  in.  wide,  which 
is  a  more  representative  size  for  a  standard  weather-radar  display. 

The  display  format  used  in  this  study  is  illustrated  in  figures  3  and  4.  The 
own-ship  symbol  representing  the  location  of  own-aircraft  was  centered  horizontally 
and  was  offset  vertically  one-third  up  from  the  bottom.  Map  information  provided  on 
the  display  gave  route  structure  and  waypoints  for  the  instrument  landing  system 
(ILS)  approach  to  runway  35R.  The  display  was  oriented  "track  up"  with  apparent  con¬ 
tinuous  movement  of  the  map  information  about  the  fixed  own-ship  symbol.  Six  map 
scales,  ranging  from  1.0  to  32.0  n.mi./in.,  were  available  to,  and  controllable  by, 
the  test  subjects. 

A  straight-line  vector  extended  from  the  own-ship  symbol  projecting  a  scaled 
distance  of  5  n.mi.  directly  ahead.  Range  arcs  were  displayed  on  the  vector  at 
scaled  ranges  of  3  n.mi.  and  5  n.mi.,  which  were  the  prescribed  spacing  intervals  for 
the  test. 

Traffic  aircraft  were  displayed  on  the  CDTI  referenced  to  the  map  display. 

Unlike  the  map,  however,  the  traffic  data  were  not  updated  continuously  but  at  4-sec 
intervals.  Between  updates,  the  traffic  symbology  would  remain  fixed  to  the  moving 
map  and  then  jump  to  its  new  position  at  the  update. 

The  traffic  symbology  was  obtained  from  reference  4  and  was  the  same  as  in  ref¬ 
erence  5.  Figure  5  illustrates  the  symbology  and  the  information  provided  the  pilot 
concerning  the  aircraft  traffic.  Aircraft  within  ±500-ft  altitude  were  considered 
"at"  own-ship  altitude.  The  trend  vector  on  the  traffic  indicated  where  the  traffic 
would  move  in  60  sec  at  its  current  groundspeed  and  heading.  The  past-position  dots 
showed  where  the  traffic  had  been,  relative  to  the  map,  on  the  previous  three  posi¬ 
tion  updates.  The  alphanumeric  data  tags  provided  identification,  absolute  altitude, 
and  groundspeed  information  for  the  traffic  (fig.  5).  The  trend  vectors,  past- 
position  dots,  and  data  tags  were  independently  selectable  by  the  test  subject  at  any 
time  during  a  run.  Selection  of  a  display  option  resulted  in  that  option  appearing 
for  all  the  displayed  traffic.  The  alphanumeric  characters  and  the  symbols  were  of 
constant  size,  independent  of  map  scale. 


Sensor  Noise  Model 

For  the  purposes  of  this  study,  sensor  noise  is  defined  as  the  random  inaccura¬ 
cies  in  the  measurement  of  the  horizontal  location  of  aircraft  traffic.  This 
measurement  is  assumed  to  be  performed  by  some  type  of  traffic  sensor  located  onboard 
the  aircraft.  Airborne  radars  and  active  or  passive  collision  avoidance  systems  with 
directional  capability  are  examples  of  airborne  traffic  sensors. 

The  measured  traffic  location  consists  of  a  distance  component  of  range  and  a 
directional  component  of  bearing.  These  components  are  measured  separately  and  have 


essentially  independent  errors  associated  with  them.  Figure  6  illustrates  the  geom¬ 
etry  of  range  and  bearing  measurements  of  traffic  location  with  errors  in  both  range 
and  bearing,  The  errors  are  assumed  to  be  normally  distributed,  high-frequency  ran¬ 
dom  noise.  Actual  measurement  of  the  location  of  a  particular  aircraft  occurs  at 
discrete  intervals.  Depending  on  the  particular  traffic  sensor,  the  length  of  time 
between  actual  measurements  is  quite  variable.  For  the  purposes  of  this  study,  the 
measurement  interval  was  chosen  to  be  4  sec,  which  is  the  approximate  interrogation 
rate  of  terminal-area  secondary  surveillance  radars. 

The  sensor  noise  model  implemented  in  this  study  functioned  as  follows.  The 
standard-deviation  values  for  both  range  and  bearing  components  of  sensor  noise  were 
predefined  at  the  start  of  each  simulated  approach.  Using  these  standard-deviation 
values,  the  components  of  range  and  bearing  error  were  calculated  at  1-sec  intervals 
by  a  random  number  generation  routine.  These  error  components  were  sampled  every 
4  sec  and  added  to  the  actual  range  and  bearing  values  for  each  of  the  aircraft. 

This  technique  produced  an  apparent  low-frequency  random  error  in  the  traffic  loca¬ 
tion  presented  on  the  cockpit  traffic  display  with  standard  deviations  equal  to  the 
preselected  values  of  sensor  noise. 


Traffic  Profiles 

lhe  traffic  scenario  utilized  in  this  study  was  taken  directly  from  reference  5. 
The  scenario  contained  aircraft  that  were  landing,  departing,  and  flying  over  the 
Denver  terminal  area.  The  flight  paths  of  the  background  traffic  simulated  pub¬ 
lished  instrument  procedures  and  hypothetical  radar  vectoring  for  take-off  and  land¬ 
ing,  utilizing  runways  35L  (left)  and  35R  (right)  in  a  parallel,  but  not  simulta¬ 
neous,  operational  manner. 

Three  new  aircraft  profiles  were  generated  for  this  study  to  be  used  as  substi¬ 
tutes  for  the  lead  aircraft  from  reference  5.  The  initial  conditions  for  each  of 
these  aircraft  were  the  same.  The  initial  position  was  at  the  Kiowa  VORTAC  (IOC), 
with  a  heading  of  253°,  an  indicated  airspeed  of  250  knots,  and  an  altitude  of 
14  000  ft.  Each  aircraft  flew  the  same  published  approach  to  runway  35R  (fig.  2); 
however,  there  were  significant  differences  in  the  speed  profiles  of  the  three  air¬ 
craft.  The  speed  profiles  as  a  function  of  distance  to  runway  threshold  are  shown  in 
figure  7.  On  the  base  leg  of  the  approach,  aircraft  1  maintained  a  fairly  constant 
indicated  airspeed  of  250  knots.  Aircraft  2  and  aircraft  3  flew  at  indicated  air¬ 
speeds  of  approximately  240  and  260  knots,  respectively.  These  airspeed  variations 
represent  the  ±1 0-knot  tolerance  which  could  accompany  an  ATC  instructed  airspeed  of 
250  knots.  After  the  turn  to  final,  the  three  aircraft  followed  different  decelera¬ 
tion  patterns,  arriving  at  the  final  approach  speed  of  130  knots  at  approximately  the 
same  location  on  the  approach  path.  The  three  deceleration  patterns  were  considered 
suitable  for  airline  operations  with  flap  and  gear  extensions  within  appropriate 
speed  limits,  and  accomplished  without  the  use  of  speed  brakes.  The  initial  position 
of  own-ship  was  7  n.mi.  behind  the  lead  aircraft  at  the  same  heading  and  altitude 
with  an  indicated  airspeed  of  290  knots.  The  traffic  identifier  for  the  lead  air¬ 
craft  was  the  same  for  all  conditions,  and  the  test  subjects  were  not  informed  of  the 
variations  in  speed  profiles  or  the  number  of  lead  aircraft  being  used  in  the  study. 


Task  Description 

The  basic  piloting  task  in  this  study  was  a  manual  instrument  approach  in  a 
terminal-area  environment  utilizing  conventional  navigation  information.  Addition- 
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ally,  the  test  subjects  were  asked  to  utilize  the  CDTI  display  to  establish  and 
maintain  specified  separation  intervals  on  a  preceding  aircraft  following  the  same 
approach  path. 

The  description  of  initial  conditions,  piloting  task,  and  performance  variables 
to  be  measured  were  given  the  test  subjects  prior  to  participating  in  the  test.  (See 
the  appendix.)  Ihe  test  subjects  were  further  instructed  to  fly  the  simulator  in  a 
manner  they  deemed  acceptable  for  airline-type  operations  and  to  avoid  radical  maneu¬ 
vers.  Besides  being  NASA  research  pilots,  the  test  subjects  had  attended  an  airline 
training  school  and  were  experienced  in  flying  the  Boeing  737  aircraft. 

As  described  previously,  the  simulator  used  for  this  study  was  a  fixed-base 
partial-workload  cockpit.  It  was,  therefore,  impossible  to  simulate  the  full- 
workload  environment  associated  with  "real-world"  operations.  Previous  ejqperience 
had  indicated  that  utilizing  the  standard  two-man  crew  in  part-task  simulations  of 
this  nature  resulted  in  unrealistically  low  workload  levels.  For  this  reason,  each 
test  subject  in  this  study  was  required  to  function  essentially  as  a  single  pilot 
performing  all  decision-making  functions  and  traffic-display  monitoring  while  exer¬ 
cising  total  manual  control  of  the  simulated  aircraft,  The  only  tasks  not  required 
of  the  test  subjects  were  manual  operation  of  landing  gear  and  flaps,  tuning 
of  radios  to  proper  navigation  frequencies,  and  changes  in  traffic  display  formats, 
these  functions  were  performed  by  the  test  engineer  at  verbal  requests  of  the  subject 
pilot. 

It  should  be  noted  that  the  pilots  were  instructed  to  "fly"  the  flight  director 
roll  and  pitch  command  bars  as  precisely  as  possible  throughout  each  approach,  this 
was  done  to  further  limit  the  amount  of  time  the  pilots  had  to  focus  on  the  traffic 
display  which  was  located  outside  the  primary  instrument  scan  area. 


Test  Conditions 

A  total  of  18  unique  combinations  of  test  variables  were  devised  for  this  study. 
Two  NASA  research  pilots  flying  3  replications  of  each  test  condition  resulted  in  a 
total  of  108  simulator  runs.  Table  I  presents  the  matrix  of  test  conditions  used  for 
both  test  subjects.  The  test-sequence  number  given  in  the  table  indicates  the  order 
in  which  the  runs  were  made.  This  order  was  randomized  with  respect  to  the  sensor 
noise  level  and  traffic  set  used  for  the  lead  aircraft.  The  entire  matrix  of  18  test 
conditions  was  completed  prior  to  any  replications. 

The  primary  independent  variables  of  interest  for  this  study  were  the  range  and 
azimuth  components  of  the  sensor  noise.  Six  values  of  azimuth  noise  were  tested  with 
no  range  noise,  and  four  values  of  range  noise  were  tested  at  two  constant  azimuth 
noise  levels.  This  setup  allowed  independent  evaluation  of  each  component  of  sensor 
noise  while  also  permitting  combinations  of  range  and  azimuth  sensor  noise  to  be 
tested. 

Three  independent  variables  of  secondary  interest  were  the  provision  of  traffic 
groundspeed  data  tags,  the  traffic  profile  used  as  lead  aircraft,  and  the  pilot. 

The  test  matrix  was  set  up  such  that  a  subset  could  be  used  in  a  2  x  2  x  2  full  fac¬ 
torial  analysis  of  variance  on  these  three  variables.  The  conditions  used  for  this 
analysis  are  indicated  in  table  I  and  detailed  in  table  II.  The  additional  lead- 
aircraft  traffic  profile  (traffic  set  1),  which  was  not  used  in  the  analysis  of  vari¬ 
ance,  was  included  in  the  study  to  help  minimize  pilot  anticipation  of  lead-aircraft 
speed  changes. 


Data  obtained  during  the  study  consisted  of  qualitative  opinion  in  the  form  of 
a  pilot  questionnaire  as  well  as  quantitative  performance  measures.  The  approach 
ground  track  flown  by  the  simulated  aircraft  was  divided  into  segments  and  gates  as 
illustrated  in  figure  8.  Essentially,  the  gates  indicated  on  the  figure  correspond 
to  the  locations  on  the  approach  path  where  the  pilots  were  instructed  to  achieve  or 
maintain  a  specific  spacing  condition. 


RESULTS  AND  DISCUSSION 
Effects  of  Sensor  Noise 

Spacing  performance.-  The  horizontal  spacing  between  own-aircraft  and  lead- 
aircraft  was  recorded  throughout  each  data  run.  The  spacing  values  at  each  of  thej 
five  gates  were  used  as  performance  measures  to  evaluate  the  effect  of  increasing 
sensor  noise  levels  on  pilot  ability  to  establish  and  maintain  a  specified  separa¬ 
tion  distance.  The  hypothesis  prior  to  undertaking  the  simulation  was  that  increas¬ 
ing  noise  levels  in  the  displayed  traffic  position  would  degrade  the  pilot  spacing 
performance  to  a  noticeable  degree.  The  noise  component  in  the  longitudinal  (range) 
direction  was  considered  to  be  the  most  critical  for  the  in-trail  spacing  task. 

Figures  9  to  13  present  the  spacing  performance  achieved  by  the  test  subjects  at 
each  of  the  five  gates  along  the  approach  path.  In  part  (a)  of  each  figure,  aircraft 
separation  is  plotted  versus  azimuth  noise  with  no  range  noise  for  target  1 ;  in 
part  (b),  aircraft  separation  is  plotted  versus  range  noise  with  1°  azimuth  noise  for 
target  2;  in  part  (c),  aircraft  separation  is  plotted  versus  range  noise  with  3° 
azimuth  noise  for  target  3.  These  data  are  presented  in  this  manner  to  isolate  the 
effects  of  the  range  and  azimuth  noise  components  on  spacing  performance.  However, 
this  test  design  did  not  allow  for  a  comparison  of  traffic  types.  The  data  points  on 
the  figures  represent  the  average  of  the  spacing  values  from  six  data  runs  (two 
pilots  with  three  replications).  The  dispersion  in  the  spacing  performance  at  each 
data  point  is  represented  by  the  la  standard-deviation  bars  shown  in  figures  9 
to  13.  Also  included  on  the  plots  is  a  dashed  line  representing  the  spacing  which 
the  test  subjects  were  instructed  to  achieve  or  maintain  at  that  location  on  the 
approach  path.  Note  that  no  spacing  was  specified  for  gate  3  and  no  reference  line 
is  presented  on  the  data  plots  for  that  gate  (fig.  11). 

As  can  be  seen  in  figures  9(a)  to  13(a),  the  plots  of  spacing  versus  azimuth 
noise  at  all  the  data  gates  reveal  nearly  constant  mean  spacing  values  with  increas¬ 
ing  azimuth  noise  for  a  particular  plot.  In  addition,  the  dispersion  about  the  data 
points  is  also  fairly  constant  with  increasing  azimuth  noise.  These  results  indicate 
that  sensor  noise  in  the  azimuthal  direction  had  negligible  effect  on  the  spacing 
performance  achieved  by  the  test  subjects  during  the  in-trail  following  task.  This 
is  not  surprising  since  spacing  is  a  longitudinal  task,  and  the  test  subjects  were 
provided  with  spacing  arcs  on  the  traffic  display  to  compensate  for  azimuthal  offsets 
as  great  as  15°. 

The  data  presented  in  parts  (b)  and  (c)  of  figures  9  to  13  indicate  no  clear 
trend  in  spacing  performance  as  a  function  of  the  range  component  of  sensor  noise. 

In  general,  the  mean  and  standard-deviation  spacing  values  show  negligible  degrada¬ 
tion  in  spacing  performance  with  increasing  levels  of  range  noise.  This  result  is 
contrary  to  the  subjective  pilot  opinion  concerning  the  effects  of  sensor  noise  as 
obtained  in  the  form  of  pilot  ratings.  Following  each  simulated  approach,  the  test 
subjects  were  asked  to  rate  the  effect  of  display  noise  on  their  spacing  performance 


during  the  run.  The  results  of  this  rating  (fig.  14)  indicated  a  consistent  trend  of 
increasing  effect  with  increasing  noise  level,  with  the  greatest  effect  corresponding 
to  the  highest  level  of  range  noise.  Despite  these  ratings,  the  spacing  data  indi¬ 
cate  no  trend  in  spacing  performance  as  a  function  of  range  noise.  The  most  reason¬ 
able  explanation  for  this  apparent  discrepancy  is  the  spacing  accuracy  which  the 
pilots  were  trying  to  achieve.  Pilot  comments  indicated  that  when  they  were  within 
approximately  0.25  n.mi.  of  the  desired  spacing  they  would  minimize  the  effort  to 
improve  the  spacing.  Hie  relatively  long  time  period  required  to  change  spacing  and 
the  increased  pilot  workload  associated  with  "fine-tuning"  the  spacing  interval  were 
cited  as  factors  contributing  to  this  tolerance  level  in  the  spacing  error.  Hiis 
spacing-error  tolerance  resulted  in  greater  spacing  errors  at  the  low  sensor  noise 
levels  than  would  have  been  possible  had  the  pilots  strived  for  more  accurate  perfor¬ 
mance.  It  is  possible  that  the  spacing  data  would  have  indicated  a  trend  of  degraded 
spacing  performance  with  increasing  range  noise  if  the  pilots  had  been  instructed  to 
reduce  their  spacing  tolerance.  It  is  significant  to  note,  however,  that  a  tolerance 
on  spacing  is  highly  desirable  from  a  workload  standpoint  as  well  as  to  dampen  possi¬ 
ble  chain  instabilities  for  multiple  in-trail  following  conditions.  It  would,  there¬ 
fore,  appear  that  actual  spacing  performance,  given  a  modest  spacing  tolerance,  is 
essentially  unaffected  by  range  noise  levels  up  to  the  maximum  tested  in  this  study. 

Flight  director  tracking  performance.-  Deviation  of  the  flight  director  command 
bars  from  the  centered  location  was  recorded  at  1-sec  intervals  during  two  segments 
of  the  approach.  Hie  first  segment  extended  along  the  base  leg  of  the  approach. 
During  this  segment,  the  navigation  radio  was  tuned  to  the  Kiowa  VORTAC,  and  the  roll 
command  deviation  on  the  flight  director  was  recorded.  Hie  second  segment  extended 
for  10  n.mi.  on  final  approach,  with  both  roll  and  pitch  command  deviations  on  the 
flight  director  being  recorded.  *  Hie  command  deviation  data  during  segments  were 
analyzed  to  obtain  a  mean  and  standard-deviation  value  for  each  run  for  both  the  roll 
and  pitch  command  basis.  Since  both  the  mean  and  standard-deviation  values  are  indi¬ 
cations  of  flight  director  command-bar  tracking  performance,  the  absolute  value  of 
the  mean  was  added  to  the  standard  deviation  to  obtain  a  single  number,  referred  to 
here  as  simply  command-bar  deviation,  to  quantify  the  performance  of  a  test  subject 
during  a  particular  run.  Hie  smaller  the  command-bar-deviation  value,  therefore,  the 
more  accurate  was  the  command-bar  tracking  performance  achieved  by  the  test  subject. 
Hiis  method  of  quantifying  command-bar  tracking  performance  was  chosen  since  it 
provides  an  equal  weighting  to  both  offsets  in  command-bar  position  as  well  as  the 
standard  deviation  of  command-bar  errors.  It  should  be  noted  that  these  data  are  not 
normally  distributed.  Standard  analysis  techniques  (t-test,  analysis  of  variance, 
etc.)  may  not  be  applicable. 

Figures  15  and  16  present  the  roll  and  pitch  tracking  performance  as  a  function 
of  azimuth  and  range  sensor  noise.  Once  again,  each  data  point  represents  the  aver¬ 
age  of  six  runs  with  the  dispersion  represented  by  the  1c  standard  deviation  bars. 

Hie  dashed  line  on  each  plot  represents  the  average  flight  director  tracking  perfor¬ 
mance  that  the  test  subjects  were  able  to  achieve  when  they  flew  the  identical 
approach  without  the  addition  of  the  CDTI  self -separation  task.  As  can  be  seen,  the 
addition  of  the  CDTI  task  degraded  the  flight  director  tracking  performance  of  the 
test  subjects.  Hie  addition  of  sensor  noise  to  traffic  display,  however,  did  not 
further  degrade  the  tracking  performance  as  might  have  been  expected.  Pilot  ratings 
of  the  effect  of  sensor  noise  on  flight  director  tracking  performance  (fig.  17)  indi¬ 
cate  a  slight  effect  of  the  sensor  noise;  however,  pilot  comments  confirmed  that  the 
addition  of  the  CDTI  spacing  task  was  the  major  workload  increase  and  accounted  for 
the  bulk  of  the  tracking  performance  degradation.  It  would  appear  that  any  increase 
in  workload  caused  by  the  increase  in  sensor  noise  level  is  minor  in  comparison  with 


the  addition  of  the  CDTI  self -separation  task  itself.  Pilot  comments  indicated  that 
mental  workload  and  confusion  were  very  much  affected  by  the  display  noise  level  even 
though  it  was  not  necessarily  reflected  in  the  flight  director  tracking  performance. 

Pilot  acceptance.-  A  subjective  evaluation  of  acceptable  levels  of  displayed 
sensor  noise  was  obtained  through  pilot  comments  and  ratings.  At  the  conclusion  of 
each  simulated  approach,  the  test  subjects  were  asked  to  rate  the  level  of  noise  they 
detected  in  the  traffic  display  using  the  following  scale: 


The  pilots  were  given  no  guidelines  as  to  the  maximum  levels  of  noise  they  would 
encounter;  however,  they  were  instructed  to  assign  a  noise-level  rating  which  they 
considered  to  be  the  maximum  acceptable  noise  they  would  tolerate  for  operational  use 
of  the  CDTI.  At  the  conclusion  of  the  tests,  one  of  the  pilots  chose  rating  2  (small 
amount  of  noise),  while  the  other  pilot  chose  rating  3  (moderate  noise)  as  the  maxi¬ 
mum  tolerable  noise  level. 

The  ratings  assigned  by  both  pilots  for  all  runs  at  the  same  actual  sensor  noise 
level  were  averaged  and  are  presented  on  a  plot  of  azimuth  noise  versus  range  noise 
in  figure  18.  The  pilots  had  no  trouble  detecting  the  relative  magnitudes  of  the 
display  noise,  as  is  evident  by  increasing  trend  in  the  ratings  with  increasing  noise 
level,  the  region  marked  "satisfactory"  on  figure  18  represents  the  sensor  noise 
levels  which  received  an  average  rating  equal  to  or  less  than  the  rating  the  pilots 
had  assigned  as  the  maximum  acceptable  noise  level  (an  average  of  2.5  for  both 
pilots).  The  region  marked  "unsatisfactory"  represents  the  noise  levels  that 
received  ratings  in  excess  of  the  maximum  acceptable  rating. 

Both  pilots  agreed  that  despite  the  unfavorable  ratings,  even  the  highest  noise 
levels  tested  did  not  prevent  them  from  accomplishing  the  in-trail  spacing  task.  The 
primary  objection  to  the  higher  noise  levels  was  the  increased  mental  workload  asso¬ 
ciated  with  visually  averaging  the  mean  position  of  the  target  aircraft  over  several 
position  updates.  The  unsatisfactory  ratings  given  the  higher  noise  levels  indicated 
the  pilots  felt  that  such  levels  of  display  noise  required  an  unacceptable  amount  of 
effort  on  the  part  of  the  pilot  to  discern  the  correct  position  of  the  traffic.  The 
ratings  were  limited  to  the  in-trail  spacing  task,  and  both  pilots  indicated  that 
other  tasks  might  well  have  higher  or  lower  threshold  levels  of  maximum  acceptable 
display  noise. 


Effects  of  Groundspeed  Display  and  Traffic  Profiles 

The  effects  on  mean  spacing  performance  of  different  lead  aircraft  speed  pro¬ 
files,  different  pilots,  and  the  displ' v  of  grour  Speed  information  were  evaluated 
using  an  analysis-of-variance  test  for  -«'*»>ifi-  ,ce  at  each  of  the  data  gates  along 


the  approach  (ref.  6).  The  data  from  8  of  the  18  test  conditions  provided  a 
2x2x2  full-factorial  matrix  with  6  replications  as  indicated  in  table  II.  It 
should  be  noted  that  test  conditions  used  in  this  matrix  contained  ensor  noise 
errors  which  are  not  included  as  factors  in  the  analysis  of  variance.  The  pooling 
of  data  incorporating  different  sensor  noise  levels  is  felt  to  be  justified,  since 
sensor  noise  was  found  to  have  a  negligible  effect  on  spacing  performance  in  the 
previous  analysis.  In  addition,  the  limited  number  of  pilots  and  traffic  profiles 
used  in  this  study  do  not  permit  a  rigorous  analysis  of  the  effects  of  these  factors 
on  spacing  performance.  The  results  from  the  current  study  must,  therefore,  be 
viewed  as  providing  trends  and  insight  into  the  importance  of  the  variables  in  con¬ 
ducting  CDTI  self -spacing  experiments,  and  care  must  be  taken  in  extrapolating  these 
results  to  more  general  conditions. 

Table  III  presents  the  computed  F-values  for  the  analysis-of -variance  tests  at 
each  of  the  five  data  gates.  The  results  in  terms  of  significant  factors  at  each 
gate  are  presented  as  follows: 


As  might  have  been  expected,  the  factor  which  consistently  appears  as  having  a 
significant  effect  on  the  spacing  performance  is  the  pilot.  The  manual  nature  of  the 
self-spacing  task  in  this  study  places  a  high  demand  on  the  pilot,  and  the  resulting 
spacing  performance  is,  therefore,  pilot  dependent  to  a  great  extent.  The  signifi¬ 
cance  of  this  result  should  not  be  minimized.  The  values  of  the  spacing  performance 
achieved  by  the  test  subjects  in  this  study,  or  any  study  involving  a  small  sample  of 
test  subjects,  should  not  be  used  to  extrapolate  to  absolute  performance  of  the  gen¬ 
eral  population  of  pilots.  Care  must  be  observed  in  selecting  an  adequate,  random 
group  of  test  subjects  if  such  extrapolation  is  desired. 

The  traffic  set  factor  is  seen  to  have  a  significant  effect  on  spacing  perfor¬ 
mance  at  gate  1.  In  figure  7,  the  speed  profiles  of  the  two  target  aircraft  used  in 
the  analysis  of  variance  (targets  2  and  3)  are  seen  to  be  approximately  20  knots 
different  during  the  base  leg  of  the  approach.  At  gate  1,  where  the  test  subjects 
were  to  initially  establish  a  5-n.mi.  spacing,  this  difference  in  speed  between  the 


two  target  aircraft  has  a  significant  effect  on  the  ability  of  the  test  subjects  to 
achieve  the  desired  spacing.  At  gate  2,  where  the  test  subjects  were  to  maintain  the 
5-mile  spacing  achieved  at  gate  1,  there  is  little  effect  of  the  different  target 
speed  profiles  on  the  pilot  spacing  performance.  On  final  approach,  where  the  speed 
profiles  of  the  two  target  aircraft  exhibit  only  minor  differences,  the  only  signifi¬ 
cant  traffic  set  effect  noted  at  gates  4  and  5  is  pilot  and  traffic  set  interaction. 
These  results  indicate  that  differences  in  lead-aircraft  speed  profiles  can  signifi¬ 
cantly  influence  self-spacing  performance,  depending  on  the  tactical  nature  of  the 
self-spacing  task,  the  amount  of  time  available  to  accomplish  the  task,  and  individ¬ 
ual  pilot  response  to  variations  in  traffic  speed  profiles.  While  these  results  are 
far  from  conclusive,  they  do  point  out  the  necessity  to  include  the  traffic  mix  and 
speed  profile  variations  when  evaluating  CDTI  self-spacing  tasks. 

Providing  the  pilots  with  groundspeed  information  is  also  seen  to  be  a  signifi¬ 
cant  factor  at  several  of  the  data  gates.  At  gate  1,  where  there  is  a  significant 
traffic  set  effect,  there  is  also  a  significant  traffic  set  and  groundspeed  interac¬ 
tion.  On  final  approach,  at  gates  4  and  5,  groundspeed  and/or  pilot  and  groundspeed 
interaction  is  a  significant  factor.  Of  particular  interest  is  gate  3,  where 
groundspeed  is  the  only  significant  factor  to  affect  spacing  performance.  At  this 
gate,  the  test  subjects  had  no  spacing  interval  specified  but  were  in  the  process 
of  closing  the  spacing  from  5  n.mi.  to  3  n.mi.  Pilot  comments  indicated  that  with¬ 
out  groundspeed  information  on  the  lead  aircraft,  the  pilots  would  have  a  tendency 
to  "overshoot"  the  desired  spacing  and  get  closer  than  the  minimum  3-n.mi.  interval. 
The  test  subjects  would  compensate  for  this  factor  by  exercising  more  caution  in 
closing  the  spacing  when  groundspeed  information  was  not  provided  on  the  lead  air¬ 
craft.  As  a  result,  the  conditions  without  groundspeed  data  tags  had  a  significantly 
greater  mean  spacing  at  gate  3.  Figure  19  presents  the  spacing  performance  with  and 
without  groundspeed  at  the  three  gates.  As  can  be  seen,  the  greater  spacing  interval 
at  gate  3  without  target  groundspeed  is  clearly  evident.  Despite  this  pilot  compen¬ 
sation,  the  tendency  for  less  than  desired  spacing  when  target  groundspeed  is  not 
provided  is  still  evident  when  the  target  crosses  the  runway  threshold  (gate  5). 


CONCLUSIONS 

A  piloted  simulation  was  conducted  to  determine  the  effect  of  traffic-sensor 
noise  on  the  use  of  an  airborne  traffic  display  for  in-trail  self -separation  during 
approach  to  landing  operations.  The  following  conclusions  are  based  on  the  results 
of  this  study: 

1 .  Displayed  traffic  position  errors  with  standard-deviation  values  up  to 

0. 3-n.mi.  range  and  8°  azimuth  had  negligible  effect  on  the  ability  of  the 
pilots  to  perform  the  self -spacing  task. 

2.  The  test  pilots  objected  to  displayed  traffic  position  errors  with  standard- 

deviation  values  greater  than  approximately  0.1 -n.mi.  range  and  2° 
azimuth.  Mental  workload  and  confusion  over  true  traffic  position  were 
cited  as  the  basis  for  objection  to  the  higher  display  errors. 

3.  Display  of  the  lead  aircraft  groundspeed  was  found  to  affect  the  mean  spacing 

performance,  especially  during  periods  of  speed  or  spacing  changes.  Pilot 
comments  cited  the  groundspeed  information  as  a  definite  aid  in  performing 
the  spacing  task. 


4.  Hie  speed  profile  of  the  lead  aircraft  was  found  to  be  a  significant  factor 
in  the  mean  spacing  performance  achieved  by  the  test  subjects.  Wie  magni¬ 
tude  of  this  effect  was  a  function  of  the  time  available  to  accomplish  the 
spacing  task  and  individual  pilot  response  to  the  variations  in  traffic 
speed  profiles. 
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APPENDIX 


INSTRUCTIONS  TO  PILOT 


Initial  Conditions 


1.  Your  aircraft  is  a  Boeing  737  located  7  n.mi.  east  of  the  Kiowa  VORTAC  at  an 
altitude  of  14  000  ft  and  an  indicated  airspeed  of  290  knots  at  a  heading  of  2538 
toward  Kiowa. 

2.  Hie  target  aircraft  is  another  737  located  over  Kiowa  at  an  altitude  of 
14  000  ft  following  the  same  253  radial  from  Kiowa. 


Pilot  Task 

1.  You  have  been  instructed  by  Denver  ATC  to  cross  Kiowa  at  14  000  ft  and  follow 
radial  253  from  Kiowa  and  intercept  localizer  and  glide  slope  for  landing  on  run¬ 
way  35R.  You  are  cleared  to  descend  from  14  000  ft  to  10  000  ft  once  you  have 
crossed  Kiowa. 

2.  The  target  aircraft  is  flying  the  same  approach.  You  have  been  instructed  to 
self -space  on  the  traffic  by  ATC.  You  are  to  close  to  a  spacing  interval  of  5  n.mi. 
on  the  target  aircraft  by  the  time  you  are  10  n.mi.  past  Kiowa.  You  are  to  maintain 
the  5-mile  spacing  until  the  target  begins  its  turn  to  final.  Once  you  have  turned 
to  final,  you  should  close  your  spacing  on  the  target  in  order  to  obtain  a  3-mile 
spacing  when  you  cross  the  outer  marker.  You  are  to  maintain  the  3-mile  spacing 
until  the  target  crosses  the  runway  threshold. 

3.  Maximum  landing  gear  and  flap  extension  speeds  for  the  737  aircraft  must  be 
observed. 


Performance  Measures 


1.  Your  ability  to  acquire  and  maintain  the  separation  interval  will  be  measured 
and  used  as  a  performance  parameter. 

2.  throughout  the  approach,  the  flight  director  should  be  flown  as  precisely  as 
possible.  Deviations  from  centered  command  bars  on  the  flight  director  will  be 
recorded  and  used  as  a  measure  of  your  performance. 
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Traffic 

set 


Traffic 
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Indicates  order  in  which  runs  were  made. 


Used  in  statistical  analysis  to  test  significance  of  traffic  set 
groundspeed,  and  pilot  effects. 


TABLE  II.-  TEST  CONDITIONS  FOR  THREE-FACTOR  ANALYSIS  OF  VARIANCE 


Test-condition 

number 

(a) 

Factor  A 

Factor  B 

Factor  C 

7  and  10 

Pilot  1 

With  groundspeed  display 

Traffic  set  2 

13  and  16 

Pilot  1 

With  groundspeed  display 

Traffic  set  3 

11  and  12 

Pilot  1 

Without  groundspeed  display 

Traffic  set  2 

17  and  18 

Pilot  1 

Without  groundspeed  display 

Traffic  set  3 

7  and  10 

Pilot  2 

With  groundspeed  display 

Traffic  set  2 

1 3  and  1 6 

Pilot  2 

With  groundspeed  display 

Traffic  set  3 

1 1  and  1 2 

Pilot  2 

Without  groundspeed  display 

Traffic  set  2 

17  and  18 

Pilot  2 

Without  groundspeed  display 

Traffic  set  3 

TABLE  III.-  COMPOTB)  P- VALUES  FOR  ANALYSTS  OF  VARIANCE 


(Location  of  gates  1  to  5  are  defined  in  figure  8) 


Degrees  of 
freedon 


Sue  of 
squares. 
(n.ei.)2 


Mean 
square , 
(n.ni. )2 


Coeputed 
F- value 


Pilot,  A 

1 

0.44 

0.44 

Groundspeed,  B 

1 

.01 

.01 

Traffic  profile,  C 

1 

2.35 

2.35 

AB 

1 

.00 

.00 

AC 

1 

.02 

.02 

BC 

1 

.39 

.39 

ABC 

1 

.07 

.07 

Error 

40 

.99 

.02 

Total 

47 

4.27 

Gate  2 

Pilot,  A 

1 

1.15 

1.15 

Groundspeed,  B 

1 

•00 

.00 

Traffic  profile,  C 

1 

.17 

.17 

AB 

1 

.01 

.01 

AC 

1 

.01 

.01 

BC 

1 

.06 

.06 

ABC 

1 

.00 

.00 

Error 

40 

2.04 

.05 

Total 

47 

3.44 

Gate  3 

Pilot,  A 

1 

0.02 

0.02 

Groundspeed,  B 

1 

1  .28 
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Figure  2.-  Approach  chart. 
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Figure  6.-  Error  geometry  associated  with  range  and  azimuth 

measurements. 
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self-spacing  performance. 
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Figure  9.-  Effect  of  sensor  noise  on  spacing  performance  at  gate  1 
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Figure  10.-  Effect  of  sensor  noise  on  spacing  performance  at  gate  2 
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Figure  12.-  Effect  of  sensor  noise  on  spacing  performance  at  gate  4 
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Figure  15.-  Effect  of  sensor  noise  on  flight  director  roll  command  tracking 

performance. 
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Figure  16.-  Effect  of  sensor  noise  on  flight  director  pitch  command  tracking 

performance. 
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Figure  18,-  Pilot  average  subjective  rating  of  displayed  noise  level 
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Figure  19.-  Spacing  performance  on  final  approach  with  and  without  target 

groundspeed  tags. 
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